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CN ' Abstract 

TimeML is an XML-based schema for annotating temporal information over 
^J , discourse. The standard has been used to annotate a variety of resources and is 

r) • followed by a number of tools, the creation of which constitute hundreds of thou- 

sands of man-hours of research work. However, the current state of resources is 
such that many are not valid, or do not produce valid output, or contain ambiguous 
or custom additions and removals. Difficulties arising from these variances were 
highlighted in the TempEval-3 exercise, which included its own extra stipulations 
over conventional TimeML as a response. 

To unify the state of current resources, and to make progress toward easy adop- 
tion of its current incarnation ISO-TimeML, this paper introduces TimeML-strict: 
a valid, unambiguous, and easy-to-process subset of TimeML. We also introduce 
p — ^ I three resources - a schema for TimeML-strict; a validator tool for TimeML-strict, 

so that one may ensure documents are in the correct form; and a repair tool that 
corrects common invalidating errors and adds disambiguating markup in order to 
convert documents from the laxer TimeML standard to TimeML-strict. 
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1 Introduction 



«_^ , TimeML (Pustejovsky et al., 2005) is an annotation scheme for the challenging task of 

C^ ■ annotation of temporal information over natural language text. Time as expressed in 

language is complex and often ambiguous, and determining how to annotate it for e.g. 
computational processing is accordingly difficult (Jaszczolt, 2009). 

Almost a decade on from its release, the TimeML schema has been adopted by hun- 
dreds of projects worldwide, and has developed into an ISO standard (Pustejovsky et al., 
2010). It is a comprehensive, expressive annotation markup language for temporal 
annotation, having been applied to significant amounts of resources and provided a 
framework for notably furthering research in temporal information extraction and un- 
derstanding of temporal semantics. Indeed, machine-readable temporal annotation 
has found applications in a wide variety of domains, including: legal (Howald, 20 11; 
Ramakrishna et al., 201 1), linguistic theory (Derczynski and Gaizauskas, 2013), ques- 
tion answering (Saquete et al., 2009; UzZaman et al., 2012a), social media and data 



management (Derczynski et al., 2013), human interface design (UzZaman et al., 201 1), 
sports coverage (Borg, 2007), transport accident analysis (Johansson et al., 2005), work- 
ing with deaf children (Arfe et al., 2009), and especially the clinical (Jung et al., 2011; 
Sun etal., 2013). 

Over time, use cases have been discovered where it is beneficial to make some 
voluntary constraints regarding how TimeML is used. Certain informal agreements 
were entered into by researchers who wished to exchange data reliably. Recently, these 
constraints have been formally applied in the 3rd international temporal evaluation ex- 
ercise, TempEval-3 (UzZaman et al., 2012b), where the corpus' followed such agree- 
ments. This paper details and crystallises these constraints into a voluntary, formal 
standard describing a subset of TimeML, called TimeML-strict. 

As well as serving as a canonical reference of the standard used in TempEval-3, 
this paper also introduces arguments for TimeML-strict, and a range of supporting 
tools. We describe a schema for TimeML-strict; a validation tool for checking whether 
a TimeML document is conformant; and a repair tool, for tightening up existing data 
such that is it compliant. Our hope is that compliant data becomes easier to auto- 
matically process, thus widening the potential user base of TimeML as well as eas- 
ing interoperability (Lee and Romary, 2010). The resources presented should also aid 
automatic conversion of legacy temporal annotation to ISO-TimeML, increasing the 
useful lifetime of community linguistic resources that represent large amounts of effort 
and investment in semantic annotation. 

The paper is structured as follows. Section 2 discusses common difficulties en- 
countered by end-users of TimeML. Section 3 describes the changes in TimeML-strict. 
Section 4 describes supplementary resources and Section 3 makes explicit some things 
that TimeML-strict does not address. We conclude in Section 6. 

2 Problems with the current state of resources 

What follows is a selection of common technical issues encountered when processing 
TimeML, followed by a requirements specification for a standard clarification. 

2.1 Validity 

As an XML format, TimeML documents can be validated by means of an XML "doc- 
ument type description", which is part of the TimeML standard. However, not all 
resources currently distributed as TimeML are valid according to their XML document 
type description (DTD). This gives many problems when loading data as XML, via e.g. 
the document object model (DOM). The inability to load TimeML documents as XML 
directly removes many of the advantages of choosing XML for the standard - such as 
ease of use with text processing frameworks (Ogren et al., 2008; Cunningham et al., 
2013) - and costs many man-days a year per researcher working with invalid data. Er- 
rors range from mis-typed element named to wrongly-encoded characters (from e.g. 
other alphabets) to SGML-valid but structurally inconsistent documents (e.g. where 
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two TIMEX3 annotations have the same "unique" ID). Many different tools have made 
parallel efforts to overcome these difficulties. However, all these efforts would not be 
required if content creators published valid data in the first place. 

2.2 Timestamps 

It is sometimes unclear what the document creation time is. TimeML's TIMEX3 func- 
tionlnDocument attribute is there to label whether a timex is publication date, creation 
date and so on - an expressive and useful part of the schema, permitting capture of 
multiple important dates which often act in two rules, both as timexes in discourse 
and as document meta-information. Often, only one of these special-function dates is 
specified (creation time or publication date), though sometimes none is, and sometimes 
more than one is. However, there is no way of defining which of these dates should 
be used as the default reference time. Having such a definition is critical to timex 
normalisation (Llorens et al., 2012) as well as to accurate replication of results. 

2.3 What to annotate 

TimeML is often used to annotate newswire documents (e.g., in the biggest TimeML 
corpus, TimeBank (Pustejovsky et al., 2003); in the TempEval-3 corpus; and also in 
the AQUAINT TimeML corpus). A lot of these are taken verbatim from the source, 
and include a preamble of metadata that is essentially gibberish - certainly not natural 
language (Section 3.2 below contains an example). This non-linguistic content often 
gets in the way of working with various NLP tools: it is often unclear how this pream- 
ble should be treated. Does one count it as a single sentence? Should headlines and 
editorial comments within it be annotated? How about numerically encoded dates that 
occur as fragments? 

2.4 Inconsistencies 

Many documents are produced that appear consistent but are difficult to process. These 
may include, for example, non-standard id labels; EVENT elements should always 
have an eid attribute that takes the form of eXX where XX is a positive integer, though 
some tools treat the field as freeform text, or omit the e. In other cases, edits to a 
document may create partial information (through both deletion and insertion of anno- 
tations). For example, after deleting an event instance, an ALINK may still use that 
event instance ID as one of its arguments. All these phenomena create speed bumps 
working with TimeML, but can be readily checked for 

2.5 Requirements 

The majority of problems above stem from the organic way in which tools and re- 
sources have sprung up over the past years, all using the framework of TimeML. We 
hope to provide a common ground - and means of reaching it - that is both TimeML- 
compatible and also very easy to work with. One goal of this standard refinement is 



to ease the process of programming with TimeML. Therefore, it should be easy to im- 
plement systems that accept TimeML-strict. For acceptance of TimeML-strict to be 
a sufficient programming requirement for working with TimeML, it is important that 
legacy data and annotations produced by legacy tools are compatible with systems that 
expect TimeML-strict. Finally, one must not constrain the expressiveness of TimeML: 
rather, the goal it to carefully preserve this expressiveness, and maximise access to 
TimeML-annotated resources. 



3 Changes new in TimeML-strict 

This section details our proposed annotations, as an addendum to the TimeML standard 
vL2. Parts of ISO-TimeML are accepted either, though the standard is not as well-used 
as TimeML, and partially in a state of flux. TimeML-strict does include some features 
of ISO-TimeML: most notably, 

• instantiating events by including e i i d and other event instance attributes within 
an EVENT label, thus eliminating the need for MAKEINSTANCES for the ma- 
jority case where events are instantiated once; 

• support event verb form and predicate attributes (vForm and pred). 

3.1 The DCT element 

This is introduced to resolve potential ambiguity regarding the document's creation 
time, that should be used as a default anchor for timexes within the document. There 
must be exactly one DCT per document. This element should enclose a single TIMEX3 
element, with no other intervening nodes (including text nodes) - see Example 1 . 

(1) <DCT><TIMEX3 functionInDocument="CREATION_TIME" 
temporalFunction="false" tid="tO" type="DATE" 
value="2013-03-22">March 22, 2013</TIMEX3></DCT> 

In the case of documents where DCT is not known, not given, or otherwise unclear, 
give an underspecified self-closing day-level timex annotation (Example 2). 

(2) <DCT><TIMEX3 tid="tO" value="XXXX-XX-XX" /></DCT> 

3.2 The TEXT element 

This is used to specify exactly the bounds of linguistic content in the document. The 
goal is to be clear which content should be considered for annotation, and allow ex- 
clusion of non-linguistic document content. Example 3 shows how this element can be 
used to exclude newswire preamble. 

(3) <TimeML> 

AP900815-0044 



AP-NR-08-15-90 1337EDT 

u i PM-GulfRdp 8thLd-Writethru 08-15 1334 

PM-Gulf Rdp, 8th Ld-Writethru, aO 605, 1368 

Saddam Seeks End To War With Iran; Bush To Urge Jordan To Close 

Port 

Eds: SUBS 28th graf pvs, Crown Prince... to CORRECT spelling of 

Hassan; pick up 29th graf pvs, 'A CBS...' 

LaserPhotos WX6, 7, XSAVl, NY5, 10, TOKl, XAAFBl, AMMl, LaserColor XAAFBl 

By CHRISTOPHER BURNS 

Associated Press Writer 

<TEXT> 

Iraq's Saddam Hussein, <EVENT eid="e5" 
class="STATE">facing</EVENT> U.S. and Arab troops at the Saudi 



</TEXT> 
</TimeML> 

Each document must have exactly one TEXT element. No particular XML hierar- 
chical relation between the TEXT and DCT elements is required; DCT may be before, 
after, within, or even contain the TEXT element (i.e. parent, child and sibling are all 
fine). 

3.3 Schema validation requirement 

All TimeML documents should include a reference to the TimeML DTD, in order to 
assist with their validation. This requirement has not been sufficient to ensure clean, 
legible XML. TimeML-strict documents must be valid. 

As well as TimeML DTD validation, TimeML-strict also requires that documents 
validate according to a more rigourous XML schema: the TimeML-strict XSD. This 
is the core strict requirement, intended to aid processing of TimeML by other tools. 
Schema compliance make transforming TimeML-strict to other formats (such as those 
following the ISO Language Annotation Framework, including ISO-TimeML) easy, 
with one-off transformation descriptions via e.g. XSLT. Following the standard en- 
ables easy conversion to ISO-TimeML whenever updates are released to the commu- 
nity (through formal XML translations, instead of text processing or intricate SAX 
event handlers and so on). 

A TimeML validation tool is made available in order to assist meeting this require- 
ment, and the corresponding XML schema file (see Section 4.2). 

3.4 No phantom element IDs 

XML schemas can enforce checks for missing references and elements. For example, 
a TLINK may reference a nonexistent event instance; elements may have malformed 
identifiers. TimeML-strict requires that every reference to an element be an identifier 
giving reference to a findable element. 



3.5 Semantics of DURING 

In his 1983 account, Allen determined a small set of distinct possible relations between 
two temporal orderings (Allen, 1983). TimeML uses this full-interval temporal relation 
set for TLlNKs. For the most part, the relations in Allen's paper and in TimeML are 
simple to match. TimeML introduces the IDENTITY relation, to distinguish between 
events (or times) that happen at the same time, and those that are also the same thing. 
However, Allen's overlap and overlap-inverse relations do not seem to be 
accounted for in TimeML. Similarly, the TimeML DURING / DURING JNV relations 
are not defined strongly in the annotation guidelines or v 1.2 specification. 

The apparent absence of Allen's overlap relations leaves a hole in TimeML's 
expressiveness. See Example 4 - there is no other temporal relation that would be 
appropriate between el and tl. The winter starts within 2012, but continues beyond 
its termination. The relation is not inclusion or simultaneity, and both BEGINS and 
ENDS (and their inverses) require a shared interval endpoint, which is also not the case 
here. 

(4) The <EVENT eid="el" eiid="eil" class="OCCURRENCE">winter</EVENT> that 
started in <TIMEX3 tid="tl" type="DATE" value="2012 ">2012</TIMEX3> 
was one of the coldest on record. 

Further, the DURING relation seems to duplicate functionality or either INCLUDES 
or SIMULTANEOUS, depending on subjective interpretation. One popular definition 
is that it should be used when an event occurs during a timex. However, this function- 
ality is already explicitly provided by SIMULTANEOUS / IS JNCLUDED. Choosing 
one of those gives a more precise relation. It is unclear why one would want to omit 
Allen's overlap functionality but include an underspecified superlabel for just one spe- 
cific circumstance. Also, the idea that some relation labels can only apply to particu- 
lar types of intervals seems unintuitive, and is a departure from the general theme in 
TimeML of abstracting events and timexes to intervals. 

Finally, it is difficult to introduce a new relType value to TimeML for these rela- 
tions; doing so breaks TimeML compatibility, and signifies a departure from TimeML, 
which is the opposite of our goal. 

In TimeML-strict, this apparent mismatch is interpreted as an oversight, and the 
two explicitly mapped as follows. 

1. TimeML DURING is equivalent to Allen's "overlap inverse (oi)": A 
DURING B is read as "A starts during B and persists beyond the end ofB " (e.g. 
overlap where A starts within B); 

2. TimeML DURINGJNV is equivalent to Allen's "overlap (o) ": A DUR- 
INGJNV B is read as "During A, B starts and persists beyond the end of A" 
(e.g. overlap where B starts within A). 

Under TimeML-strict, for the intervals in Example 4, one might annotate: 

<TLINK lid="ll" eventInstanceID="eil" relType="DURING" 
relatedToTime="tl" /> 



This change is perhaps in conflict with some prior interpretations of TimeML, but 
it is the only way in which TimeML's relation types can be made to cover the full set 
of interval relation configurations and removes two arguably redundant links, while 
staying valid. 

As this change risks taking a departure from some conventions, one should pay 
regard to impact on existing resources. Brief examination of TimeBank 1.2 suggested 
annotator ambiguity between all of the INCLUDES, SIMULTANEOUS and DURING 
link relation types. We do not specify how to deal with such annotation, and recom- 
mend that annotations (especially in resources) be treated as slightly fuzzy, as per the 
TimeML recommendation. 

Existing closure tools should be readily modifiable to meet this specification change 
and indeed some have already been using this interpretation for a few years. 

4 Resources provided 

With TimeML-strict, three resources are made available for working with documents 
and the specification. 

4.1 XML Schema Definition 

Core to TimeML-strict is a formal validation schema. This is substantially different 
from TimeML L2's XML schema definition (XSD), building in stricter checks. There 
are two key scenarios for this schema's use: to help TimeML producers be sure that 
they have generated shareable, legible TimeML; and to enable those trying to read 
TimeML to be aware of the expected breadth of expression and level of data consis- 
tency. 

This schema includes TIMEX3 support. There is no single reference point for 
the TIMEX3 standard, but rather TimeML builds on earlier TIMEX standards. The 
TimeML-strict schema incorporates the TIMEX2 spec from Ferro et al. (2005) and the 
appropriate adaptations in TimeML L2. The schema is included with the TimeML 
validator. 

4.2 TimeML validator 

A Java validation tool that verifies whether or not documents are acceptable as TimeML- 
strict. For documents that are not valid, information is given to allow content creators 
to find the source of the problems. This tool is available via github.- 

4.3 Automatic migration and repair of TimeML 

To ease adoption of TimeML-strict, and to convert potentially invalid data into a con- 
sistent, valid format, a TimeML repair tool is made available. This converts and fixes 
common mistakes in older TimeML, and enables older resources to be processed by 
newer tools. 



See https://github.com/hllorens/TimeML-validator 



The DCT and TEXT elements are automatically added by the tool, if possible, 
which also attempts to rectify several common mistakes such as invalidly-structured ID 
strings or references to missing entities. As TimeBank vl.2 isn't valid under TimeML- 
strict, this tool is absolutely critical. The tool has also been tested with a few com- 
mon TimeML-generating systems and can rectify their output, repairing both invalid 
TimeML and also adding requisite information to make the output TimeML-strict com- 
pliant. It is accessible via open-source repository.'' 

5 Beyond the scope of TimeML-strict 

This paper has so far considered problems encountered "in the wild" and proposed 
fixes for them. There are also those problems that are deliberately not addressed by 
TimeML-strict. This section introduces a few of these phenomena with explanations 
of why TimeML-strict does not constraint against them. 

The extent of timex and event elements should be just a single word, according to 
the TimeML English annotation guidelines. However, there are many cases where this 
is not possible, especially with timexes, where qualifying words are critical (e.g. last 
in "last year". Also, some resources migrated from older standards include slightly 
longer words (Derczynski et al., 2012). Making this invalid in TimeML-strict would 
be an unreasonable constraint and would reduce the expressiveness of TimeML, while 
offering little benefit to the technical process (the annotated text is e.g. a single DOM 
node regardless of word count). 

TimeML-strict also does not tackle temporal consistency. The transitive nature of 
many interval relation types means that it is possible to create an annotation that is 
inconsistent (Example 5). 

(5) <TLINK lid="ll" eventInstanceID="eil" relType="BEFORE" 
relatedToEventInstance="ei2" /> 
<TLINK lid="12" event InstanceID="eil " relType="INCLUDES" 
relatedToEventInstance-"ei2 " /> 

Although this presents difficulties when performing temporal closure or attempting 
to use inference-based learning for relation labelling, such annotations remain valid. 
TimeML should be capable of annotating any document. In the case of newswire, 
one should not make the assumption that the linguistic utterances of journalists are 
temporally consistent in the first place. 

It is also possible to create somewhat "orphaned" annotations, e.g. uninstantiated 
events. There is no requirement to instantiate events, and inclusion of any event at- 
tribute other than its class is optional. Indeed, TimeML's possible values for part of 
speech, tense, aspect and so on may be viewed as recommendations rather than dec- 
larations of the best way to annotate this values for temporal information processing, 
reducing how critical this information is. 

Some of these kinds of meta-consistency is explicitly checked for by the CAVaT 
tool Derczynski and Gaizauskas (2010), which includes a selection of modules for val- 
idating TimeML. However, they are all permissible under TimeML-strict. 



See https://bitbucket.org/leondz/timeml-repair 



6 Conclusion 

This paper has detailed a refinement of the TimeML standard, hoping to bring together 
the many diverse outputs of the successful ongoing TimeML project and make life eas- 
ier for those working with TimeML data. Along with the abstract parts of the refine- 
ment, three concrete tools are presented: a schema; a validation tool; and a repair tool. 
This is an iterative, voluntary step, which has the added benefit of preparing for rapid 
and seamless transition between TimeML and ISO-TimeML. TimeML-strict offers a 
common base for simpler computing with temporal annotations, allowing interested 
researchers to get on with experimentation and discovery. 
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